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(54) Title: DEVICE FOR DEMULTIPLEXING CODED DATA 
(57) Abstract 

The invention relates to a device for demultiplexing coded data 
corresponding to multimedia objects. The data are organized in a 
bitstream divided into individually accessible portions - for instance 
Access Units for an MPEG-4 compliant bitstream - th^selves 
subdivided into smaller data entities - Sl^packets in die cited 
example - which all the demultiplexing operations are applied to. 
Said operations are mainly, for all the data received at the input of the 
demultiplexing device: reading and formatting of a buffer, extraction 
of successive Access Unit segments, progressive reconstruction of a 
complete Access Unit, transmission of said Access Unit to an output 
buffer or file. 
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he invention relates to a device for demultiplexing coded data available in the 
form of a bitstream corresponding to multimedia objects, and more particularly in the form of 
a bitstream compliant with the MPEG-4 standard. 

The MPEG-4 standard provides standardized ways to represent audio-visual 
5 objects (called AVOs) of natural or synthetic origin, to compose them together to create 
compound AVOs that form audio- visual scenes, to multiplex and synchronize the data 
associated with AVOs, and to interact with the audio-visual scenes generated or reconstmcted 
at the receiver's end. 

It may be recalled that, as shown in Fig.l, an audio- visual scene, received by an 

10 MPEG-4 system such as described for instance in the document "MPEG-4 : context and 
objectives'', R.Koenen and al.. Signal Processing : Image Communication 9 (1997), May 
1997, n'*4, pp.295-304, is generally composed of several AVOs organized in a hierarchical 
fashion. The leaves of this hierarchical organization are primitive AVOs such as, in the case of 
the illustrated audiovisual presentation : a background, the picture of a talking person, the 

15 voice associated with that person, a 3D object present in the scene, and so on (of any type - 
text, graphics,...- and which may be either bi- or tridimensional). Fig.l also illustrates 
schematically how video and audio compositors process the scene before it is rendered and 
displayed for the user, who has then the possibility to interact with it. 

The data associated with these AVOs are conveyed in one or more Elementary 

20 Streams (ESs), characterized by the quality of service they require for transmission and some 
other parameters. The so-called MPEG-4 System Layer Model, schematically depicted in 
Fig.2, provides a common model on which all implementations of MPEG-4 terminals can be 
based. As shown in Fig,3 that illustrates the major components of such an MPEG-4 terminal 
(at the receiver side), data streams coming from a network or a storage medium 

25 ('Transmission/Storage Medium", in Fig.2 ; "Network Layer", in Fig.3) in the form of 

TransMux Streams must be properly demultiplexed to recover the Elementary Streams. These 
Elementary Streams are then passed to the appropriate decoders in view of their 
decompression, and in order to reconstruct the original AVO (Primitive AV Objects). Decoded 
AVOs, along with scene description indications giving information on the composition of the 
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concerned scene, are then used to compose and render the scene as described by its author (in 
a given hierarchical, interactive form). Also to the extent allowed by the author, upstream data 
(user events, class request,. . .) are sent back to the Network Layer in order to interact with the 
scene. 

5 More precisely, the MPEG-4 terminal depicted in Fig.2 is a multi-layer 

structure consisting of the three following layers : a TransMux layer 21, a FlexMux layer 22 
and an Access Unit layer 23. 

The TransMux layer 21, which designates any existing or future underlying 
multiplex functionality that is suitable to transport MPEG-4 data streams (thus allowing 

10 MPEG-4 to be used in a wide variety of operation environments), consists of a protection 
sublayer and a multiplexing sublayer (although it may not be possible to separately identify 
these sublayers in some TransMux instances, the protection sublayer has a specific interest for 
providing error protection and eiror detection tools suitable for the given network or storage 
medium). This layer, not defined in the context of MPEG-4, is in fact an interface to the 

15 network or the storage medium and allows to offer transport services matching the requested 
Quality of Service (QoS). 

The FlexMux layer 22, completely specified by MPEG-4, consists of a flexible 
tool for interleaving data (one or more Elementary Streams into one FlexMux stream) and 
allows to identify the different channels for the data that have been multiplexed, 

20 The Access Unit layer 23 allows for an identification of Access Units (video or 

audio frames, scene description commands,.. .) in Elementary Streams and a recovery of time 
base. It conveys therefore both time base information and time stamped Access Units of these 
Elementary Streams. It may however be noticed that each layer of the MPEG-4 demultiplexer 
of the terminal has to process successively the whole stream. 

25 A first object of the invention is to propose a more general implementation of 

such a demultiplexer, allowing a full integration with the subsequent decoder. 

To this end, the invention relates to a device such as mentioned in the preamble 
of the description and which is moreover characterized in that said bitstream is 
subdivided into individually accessible portions of the coded representation of said multimedia 

30 objects and all the demultiplexing operations carried out in said device are, before the 

decoding operations, successively applied, after a further subdivision of said portions into 
smaller basic data entities, to each of said small data entities. 

Another object of the invention is, more particularly, to propose a demultiplexer 
allowing to maintain a fully MPEG-4 compliant architecture. 
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To this end the invention relates to a demultiplexing device in which said 
bitstream is compliant with the MPEG-4 standard and subdivided into so-called Access Units, 
said demultiplexing operations being successively applied to smaller data segments, called SL- 
packets, of these Access Units. 
5 More precisely, the main steps of the demultiplexing procedure, applied to each 

of the successive data stmctures, will then be the following ones : 

(a) reading and formatting of a buffer ; 

(b) extraction of segments of an Access Unit ; 

(c) test of complete reconstruction, or not, of an Access Unit from the previously 
10 extracted segments ; 

(d) transmission of each complete Access Unit to an output buffer or file ; 

(e) repetition of steps (a) to (d) until no more data are present at the input of the 
demultiplexer. 

15 

The particularities and advantages of the invention will be now explained in a more detailed 
manner in relation with the following description and the accompanying drawings, in which : 

- Fig.l illustrates an example of an object-based multimedia scene, as processed 
according to the MPEG-4 standard ; 

20 — Fig,2 depicts an MPEG-4 terminal structure allowing to process and reconstruct an 

audiovisual interactive scene, after a transmission and/or a storage of the 
corresponding MPEG-4 coded data ; 

- Fig.3 shows the major components of an MPEG-4 terminal, at the receiver side ; 

- Fig.4 illustrates the steps implemented in a demultiplexing device according to the 
25 invention. 

Although the invention may be applied in a wider field of applications, it will 
be described in the case of the MPEG-4 standard. According to the MPEG-4 specifications, 
representations of multimedia objects are conveyed from source entities to destination entities 
in separate elementary streams. A specific layer -called Synchronization Layer, or Sync 
30 Layer, or SL- allows to adapt elementary stream data for communication and therefore 
provides inter alia timing and synchronization information. This layer provides for an 
encapsulation of each elementary stream, i.e. divides each elementary stream into one or more 
Access Units, which are individually accessible portions of the coded representation of a 
multimedia object and are the smallest data entities to which timing information can be 
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attributed. An Access Unit may be itself divided into small basic data entities called SL- 
Packets each of which consists of a configurable header and a payload. 

In implementations proposed up to now, each layer of an MPEG-4 
demultiplexer processes successively the whole stream, such as described for instance in the 
European patent application n°98400160.2 (PHF98505). According to the invention, an 
MPEG-4 terminal with a real-time performance may be obtained if the demultiplexing 
operations are now processed for every piece of data the MPEG-4 demultiplexer receives. 
Basically, the different steps of the demultiplexing procedure are then the following ones, as 
depicted in Fig.4 : 

(a) initialization of TransMux and FlexMux audiovisual objects ; 

(b) a buffer of a size corresponding to a payload length is read (for instance, this size 
may be of 184 bytes for the payload length of an MPEG-2 Transport Stream 
Packet) ; 

(c) said buffer (that has been read) is then formatted to a FlexMux Protocol Data Unit 
(FMPDU) by the TransMux layer ; 

(d) the FlexMux layer receives said FMPDU and extracts the corresponding 
Synchronization Layer Protocol Data Unit (or SLPDU, such an SLPDU 
representing generally a segment of an Access Unit, as said above) ; 

(e) the next step is a test : has an Access Unit been completely reconstructed from the 
previous concerned SLPDU segments, or not ? : 

(el) f an Access Unit has not been completely reconstructed, the MPEG-4 

demultiplexer stores the data into a local buffer and re-starts the 

process with another input packet ; 
(e2) f a complete Access Unit, composed of all the concerned previous 

SLPDUs, is present, it is delivered to an output buffer or written into a 

file. 

Said process is then re-initialized until no more data are present at the input of 
the demultiplexer. 

The described demultiplexing device can be easily integrated on a 
programmable multimedia processor such as the processor designated with the reference TM- 
1000 (=TriMedia) manufactured by the company Philips, and its architecture is fully MPEG-4 
compliant on said processor. 
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CLAIMS: 



1. A device for demultiplexing coded data available in the form of a bitstream 
corresponding to multimedia objects, characterized in that said bitstream is subdivided into 
individually accessible portions of the coded representation of said multimedia objects and all 
the demultiplexing operations carried out in said device are, before the decoding operations, 

5 successively applied, after a further subdivision of said portions into smaller basic data 
entities, to each of said small data entities. 

2. A device according to claim 1, wherein said bitstream is compliant with the 
MPEG-4 standard and subdivided into so-called Access Units, said demultiplexing operations 

10 being successively applied to smaller data segments, called SL-packets, of these Access Units. 

3. A device according to claim 2, characterized in that said demultiplexing 
operations are, after an initialization of the elements of the demultiplexing device, carried out 
according to the following procedure : 

(a) reading and formatting of a buffer ; 

(b) extraction of segments of an Access Unit ; 

(c) test of complete reconstruction, or not, of an Access Unit from the previously 
extracted segments ; 

(d) transmission of each complete Access Unit to an output buffer or file ; 

(e) repetition of steps (a) to (d) until no more data are present at the input of the 
demultiplexer. 
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